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TITLE: VIDEO MAIL SERVER WITH REDUCED FRAME LOSS. 

Technical Field 

The present invention relates to video telephony, and in particular to a video mail 
5 server and methods for recording and playing back video mail messages. 

Background of the Invention 

For quite some time telephone systems, including both private telephone 
systems, wireless telephone service provider systems, and PSTN service provider 

10 systems have included voice mail systems. 

The voice mail functions of a telephone system are typically provided by a voice 
mail server. Upon the occurrence of certain events, such as a telephone line being 
busy or unanswered, the telephone system will signal the call (in a process typically 
referred to as roll-over" to the voice mail server. The voice mail server will receive the 

15 telephone call thereby opening a recording session. 

During the recording session, the voice mail server will prompt the caller to leave 
a message, capture the audio stream from the caller, and store the captured audio 
stream for subsequent play back to the voice mail box owner. The recorded message is 
typically stored in digital form on magnetic media. 

20 At some later point in time, the voice mail box owner may call the voice mail 

server to establish a play back session. During the play back session, the server will 
prompt the voice mail box owner to authenticate him or her self, retrieve the stored 
message, and generate an audio stream to the telephone system from which the voice 
mail box owner called into the server. 

25 If the voice mail server is part of a circuit switched telephone system, both the 

recording session and the play back session will take place over circuit switched 
channels. During recording, the voice mail server will capture the analog audio or digital 
audio from the circuit switched channel. Digital audio can be stored in its received form 
and analog audio is readily digitized using known A/D converter systems. 

30 If the voice mail server is part of a VoIP telephone system, both the recording 
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session and the play back session will take place over IP channels. More specifically, 
when setting up the recording session, the server and the remote VoIP device (used by 
the caller) will negotiate a specific compression algorithm. Then, during the recording 
session, the voice mail server will receive a sequence of RTP packets over a UDP/IP 
5 channel. Each RTP packet includes one or more audio frames compressed using the 
negotiated compression algorithm. Each compressed audio frame represents a fixed 
time interval (on the order of 10 milliseconds) of digital audio. The server sequences 
and decompresses each compressed audio frame to regenerate digital audio for 
storage. 

10 At a later time, when setting up the play back session, the server and the remote 

VoIP device (used by the voice mail box user) will negotiate a specific compression 
algorithm - which may be different than the compression algorithm used during the 
recording session. Then, during the play back session, the voice mail server will 
generate a sequence of RTP packets for sending to the remote VoIP device over a 

15 UDP/IP channel. Each RTP packet includes one or more audio frames compressed 
using the negotiated compression algorithm. The remote device sequences and 
decompresses each compressed audio frame to generate acoustic audio for the voice 
mail box user. 

If the system is a hybrid type of system, it may support both: i) VoIP recording 
20 sessions and play back sessions; and ii) circuit switched recording and play back 

sessions. In such an embodiment, the session type of the recording session (e.g. VoIP 
or circuit switched) does not need to match the session type of the play back session. 

In a separate field of development, technology for transmitting motion video over 
IP networks has been developed. During an IP video session, a transmitting video IP 
25 device will receive a sequence of image frames from a video camera, compress each 
image frame of the sequence, and transmit each compressed image frame to a 
receiving video IP device. 

The receiving video IP device will decompress each compressed image frame of 
the sequence, and sequentially display each image frame of the sequence, to generate 
30 a motion video display. 
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The International Telecommunication Union (ITU) has recommended the H.263 
standard entitled Video Coding for Low Bit Rate Communications as a standard for 
compression motion video for telephony. The existence of a standard facilitates the 
development of video IP devices. 
5 There exist significant differences between transmitting motion video over an IP 

channel and transmitting audio over an IP channel. First, each compressed audio 
frame represents digital audio over a short duration of time. Each compressed video 
frame represents an image of the video stream at a fixed instant in time. Secondly, 
each audio frame is encapsulated in a single RTP packet and can be decompressed 

10 without reference to any other frames. The significance of this is that if an audio frame 
is lost in transmission, the audio for the duration of time represented by the lost frame is 
lost, but neither the preceding audio frame nor the following audio frame is affected. 

On the other hand, sequential video frames are mostly interdependent. Each 
video frame is compressed into independent frames and dependent frames. The 

15 independent frames, referred to as "intra frames" or "i-frames", can be decompressed to 
generate a complete image frame without reference to a preceding frame or a following 
frame. The dependent frames can be of two types. The first type may be referred to as 
"predictive frames" or "p-frames". A p-frame represents the difference between such 
frame and the preceding p-frame or i-frame. As such, a p-frame can not be 

20 decompressed without reference to the preceding p-frame or i-frame. The second type 
may be referred to as "bi-directional frames" or "b-frames". A b-frame can only be 
decompressed with reference to one or more preceding i/p frames and optionally one or 
more following i/p frames. 

The ratios between i-frames, p-frames, and b-frames is not fixed, but is 

25 dependent on encoding algorithms and dependent on the video image content - or 
scene. As such, the quantity of dependent frames between independent frames varies 
based on the video content. As such, the time duration between independent frames 
also varies based on video content. 

A problem with interdependent frames is that if a frame is lost during 

30 transmission, all subsequent frames that depend thereon are also lost. The image on 
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the receiving IP device will freeze until the next independent frame is received. Frame 
loss is further exacerbated by the fact that each image frame can be transmitted in 
multiple RTP packets. Loss of any RTP packet will cause the loss of the entire frame. 

It has been proposed to combine video transmissions with voice mail server 
5 technology to provide video mail services. A problem exists in that implementing a 
video mail server causes the video message to be transmitted twice. The first 
transmission is from the caller to the video mail server. The second transmission is 
from the video mail server to a user "calling in" to retrieve video mail messages. 

As such, frame loss exposure is doubled. Frames can be lost in the transmission 
10 from the caller to the voice mail server resulting in freeze frame periods stored on the 
voice mail server. When transmitted to the user, the freeze frame periods that already 
exist at the sever will be sent to the user and, in addition, further freeze frame periods 
will be created due to lost frames in the transmission to the end user. 

As such, a need exists for a video mail solution that operates in conjunction with 
15 legacy video IP devices and does not suffer the disadvantages and impracticalities 
discussed above. 

Summary of the Invention 

A first aspect of the present invention is to provide a video mail server. The 
20 video mail server comprises a video call signaling module coupled to an internet 
protocol (IP) network via an internet protocol services module. 

The video call signaling modules: i) establishes a first internet protocol channel 
with a caller remote internet video device (e.g. a caller system) to support a recording 
session over the IP network; and ii) establishes a second internet protocol channel with 
25 a user remote internet video device (e.g. a user system) to support a playback session 
over the IP network. 

A media interface is also coupled to the IP network via the internet protocol 
services module. The media interface comprises a recording module and a playback 
module. 

30 The recording module obtains a recording sequence of compressed images, that 
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represent motion video, from the caller system and stores a video mail file representing 
the recording sequence of compressed images. 

The play back module retrieves the video mail file and transferring contents of 
the video mail file, as the playback sequence of compressed images, to the user 
5 system. 

Each compressed image frame within the video mail file is one of: i) an 
independent frame from which a video image frame can be recovered utilizing only the 
independent frame; and ii) a dependent frame from which the video image frame can 
only be recovered utilizing both the dependent frame and an independent frame 
10 preceding the dependent frame in the sequence. 

In one embodiment, the video mail server may further include a video codec 
coupled to the media interface. The video codec comprises a decoder module and an 
encoder module. 

The decoder module receives the recording sequence of compressed images 
15 from the recording module and decodes the recording sequence of compressed images 
to generate motion video images. The encoder module then encodes the motion video 
images into the playback sequence of compressed images. 

The playback sequence of compressed images may be in a robust format. The 
robust format requires that at least one independent frame be included within each fixed 
20 time duration. Further, the robust format may require that the duration of time between 
independent frames be a fixed period on the order of one second. 

In one sub embodiment, the video codec operates during the recording session 
and the playback sequence of compressed images is transferred to the media interface 
for storing as the video mail file. In which case, the contents of the video mail file may 
25 be transferred directly to the user system during a play back session. 

In a second sub embodiment, the decoder module operates gets the recording 
sequence of compressed images from the video mail file during the play back session. 
And, the playback sequence of compressed images is transferred directly to the user 
system. 

30 As an alternative to encoding in a robust format, the video codec may operate 
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during the playback session in a traditional manner. Except, during the playback 
session, the playback module may receive a lost frame message from the user system 
when the user system detects loss of a frame within the playback sequence of 
compressed images. 

5 In response to receiving a lost frame message, the video codec may compress 

the next image frame of the motion video images as an independent frame to reduce 
freeze frame time at the user system. 

As a second alternative to encoding in a robust format, the video codec may 
operate during the playback session for: i) receiving the recording sequence of 

10 compressed images from the playback module; ii) decoding the recording sequence of 
compressed images to generate motion video images; and iii) queuing each motion 
video image for encoding, by the encoding module, as a lost frame correction frame. 

In this second alternative, the playback module comprises a delay buffer (24) for 
delaying the playback sequence of compressed images for a period of time such that 

15 each frame within the playback sequence of compressed images is queued for sending 
to the user system at a time that corresponds to the motion video image queued for 
encoding as a lost frame correction frame. As such a lost frame correction frame may 
be substituted for a frame in the playback sequence of compressed images in response 
to receiving a lost frame message. 

20 As an alternative of having a video codec, the call signaling module may 

establish each of the first internet protocol channel and the second internet protocol 
channel over a TCP/IP connection. As such, the internet protocol services module 
would operate TCP/IP protocols to effect re-transmission of any lost TCP/IP frames on 
each of the first TCP/IP connection and the second TCP/IP connection. In this 

25 embodiment, the playback sequence of compressed images would be the same as 
recording sequence of compressed images. 

As an alternative to establishing both the first internet protocol channel and the 
second internet protocol channel over TCP/IP connections, the call signaling module 
may establish only the first internet protocol channel over a TCP/IP connection and may 

30 establish second internet protocol channel over a UDP/IP channel. 
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In this embodiment, the video mail file would comprise the recording sequence of 
compressed images and a video codec would operate during the playback session 
using any of the schemes listed above. 

As yet another alternative to establishing both the first internet protocol channel 
5 and the second internet protocol channel over TCP/IP connections, the call signaling 
module may establish only the second internet protocol channel over a TCP/IP 
connection and may establish the first internet protocol channel over a UDP/IP channel. 

In this embodiment, the video mail file would comprise the recording sequence of 
compressed images and the recording sequence of compressed images would be 
10 transferred to the user system as the playback sequence of compressed images for the 
second internet protocol channel over TCP/IP. 

For a better understanding of the present invention, together with other and 
further aspects thereof, reference is made to the following description in conjunction 
with the accompanying drawings. The scope of the present invention is set forth in the 
15 appended clams. 

Brief Description of the Drawings 

Figure 1 is a block diagram representing a system for providing VoIP 
communication services and Internet data connectivity over a frame switched network in 
20 accordance with one embodiment of the present invention; 

Figure 2a is a ladder diagram representing set up of a recording session in 
accordance with one embodiment of the present invention; 

Figure 2b is a ladder diagram representing set up of a play back session in 
accordance with one embodiment of the present invention; 
25 Figure 3a is a flow chart representing one embodiment of operation of a 

recording module of the present invention; 

Figure 3b is a flow chart representing one embodiment of operation of a 
recording module of the present invention; 

Figure 3c is a flow chart representing one embodiment of operation of a 
30 recording module of the present invention; 
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Figure 4a is a flow chart representing one embodiment of operation of a playback 
module of the present invention; 

Figure 4b is a flow chart representing one embodiment of operation of a playback 
module of the present invention; 
5 Figure 4c is a flow chart representing one embodiment of operation of a playback 

module of the present invention; 

Figure 5a is a diagram representing a sequence of compressed video frames in 
accordance with one embodiment of the present invention; 

Figure 5b is a diagram representing a sequence of compressed video frames in 
10 accordance with one embodiment of the present invention; 

Figure 5c is a diagram representing an exemplary mode of operation of the 
present invention; and 

Figure 5d is a diagram representing an exemplary mode of operation of the 
present invention. 

15 

Detailed Description of the Exemplary Embodiments 

The present invention will now be described in detail with reference to the 
drawings. In the drawings, each element with a reference number is similar to other 
elements with the same reference number independent of any letter designation 

20 following the reference number. In the text, a reference number with a specific letter 
designation following the reference number refers to the specific element with the 
number and letter designation and a reference number without a specific letter 
designation refers to all elements with the same reference number independent of any 
letter designation following the reference number in the drawings. 

25 It should also be appreciated that many of the elements discussed in this 

specification may be implemented in a hardware circuit(s), a processor executing 
software code, or a combination of a hardware circuit(s) and a processor or control 
block of an integrated circuit executing machine readable code. As such, the term 
circuit, module, server, or other equivalent description of an element as used throughout 

30 this specification is intended to encompass a hardware circuit (whether discrete 
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elements or an integrated circuit block), a processor or control block executing code, or 
a combination of a hardware circuit(s) and a processor and/or control block executing 
code. 

Figure 1 represents a video mail server 10 coupled to an internet protocol (IP) 
5 network 12 for participating in a video mail recording session with a caller system 14 
and participating in play back session with a user system 16. 

The video mail server 1 0 comprises a network interface 30 coupling the server to 
the IP network 12, an internet protocol services module 32, a signaling client 34, a 
media interface 42, a delay circuit 24, a video coder/decoder circuit (CODEC) 38, and a 
10 data storage 28. Each of these components will be discussed in more detail herein. 

The storage 28 may be known combination of magnetic media, reading and 
writing hardware circuits, and drivers for storing data files. For example, the storage 28 
may be a known disk drive system or a combination of disk drive systems. 

The network interface 30 may be a known combination of hardware circuits and 
15 software drivers for implementing the physical layer and data link layer networking 
functions of the OSI networking model. In operation, the network interface 30 couples 
the server 10 to the IP network 12. 

The internet protocol (IP) services module 32 couples to the network interface 20 
and may be a known combination of software modules for implementing the TCP/IP 
20 protocols and the UDP/IP protocols of the network layer and transport layer of the OSI 
model. 

The signaling client 34 couples to the IP services module 32 and may operate a 
known signaling protocol for signaling and establishing real time video media session 
with remote signaling clients operating on remote devices. Exemplary signaling 

25 protocols include the Session Initiation Protocol (SIP) as promulgated by the Internet 
Engineering Task Force (IETF), the Multi-Media Gateway Control Protocol (MGCP) also 
promulgated by the IETF, and the Q.931 and the H.245 protocols of the H.323 protocol 
promulgated by the International Telecommunication Union. In further discussion of the 
signaling client 34, the SIP protocols will be used as an example, and such use is not 

30 intended to limit the scope of the invention. 
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The media interface 42 couples to the IP services module 32 and includes a 
recording module 48 and a play back module 50. 

The delay buffer 24 is coupled to the media interface 42 and may be a known 
first-in-first-out (FIFO) register or memory circuit for delaying a stream of data by a fixed 
5 duration of time 1 26 (Figure 5d). 

The video codec 38 is coupled to the media interface 42 and may be a known 
combination of digital signal processing circuits and drivers for converting between a 
sequence of motion video images and a sequence of compressed images frames 
representative thereof. 

10 As an overview, the recording module 48 obtains a recording sequence of 

compressed images 52 representing motion video provided by the caller system 14 over 
a first internet protocol (IP) channel 18 established by the signaling module 32 (in 
response to call signaling being provided to the video mail server 10 by the caller 
system 14). A representation of the recording sequence of compressed images 52 is 

15 stored as a video mail file 22 in the storage 28. 

The playback module 50 obtains the video mail file 22 and provides a playback 
sequence of compressed images 58 to the user system 16 over a second IP channel 20 
established by the signaling module 32. A more detailed discussion of the media 
interface 42 is included herein. 

20 Each compressed image within each of the recording sequence of compressed 

images 52 and the play back sequence of compressed images 58 comprises is one of 
an independent frame 54 and a dependent frame 56. An independent frame 54 
comprises a compressed representation of the complete video image. An independent 
frame 54 can be decompressed to generate the video image without reference to a 

25 preceding frame or a following frame. An exemplary independent frame 54 is the "intra 
frame" format or "i-frame" as defined in the H.263 standard, entitled "Video Coding for 
Low Bit Rate Communication", promulgated by the International Telecommunication 
Union. 

A dependent frame 56 comprises data representing the differences between the 
30 dependent frame 56 and at least one preceding independent frame 54 or dependent 
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frame 56 - and optionally one following independent frame 54 or dependent frame 56. 
A dependent frame 56 can only be decompressed with reference to the at least one 
preceding frame or following frame from which the differences deviate. Exemplary 
dependent frame formats include the "predicted frame" or "p-frame" and the "bi- 
5 directional" or "b-frame" each defined in the H.263 standard. 

As stated above, a problem with transferring a video image using a compression 
scheme with interdependent frames is that if a frame is lost during transmission, all 
subsequent frames that depend thereon are also lost. The image on the receiving IP 
device will freeze until the next independent frame 56 is received. 

10 Further, because the time period between independent frames 56 varies based 

on video image content and is not predictable, the length of freeze frame could be quite 
long. In order to reduce frame loss and thereby reduce the freeze time periods when 
the motion video is viewed on the user system 16, the server 10 implements one of the 
following frame loss embodiments. 

15 The first frame loss embodiment includes storing the recording sequence of 

compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. The video mail file 22 will be missing any 
frames that were lost during the recording session. 

During playback, the recording sequence of compressed images 52 is retrieved 

20 from the video mail file 22, transferred to the codec 38 for decoding and re-encoding in 
a robust format. The robust format requires that at least one independent frame be 
encoded for every fixed period of time. For example, the robust format may require that 
at least one independent frame be encoded for each one second time period or each 
half second time period. While any freeze frame caused by loss of framed during the 

25 recording session will still remain when viewed on the user system 16, any additional 
freeze frame caused by frame loss during the play back session will be limited to only 
one half second. 

The second frame loss embodiment is very similar to the first frame loss 
embodiment, except that the robust format requires a fixed period of time between each 
30 independent frame 56. For example, the robust format may require that an independent 
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frame 56 be generated every half second or every second. 

The third frame loss embodiment includes storing the recording sequence of 
compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. Again, the video mail file 22 will be missing any 
5 frames that were lost during the recording session. 

During playback, the recording sequence of compressed images 52 is retrieved 
from the video mail file 22, transferred to the codec 38 for decoding and re-encoding in 
a normal format. If, during the playback session, the user system 16 detects any frame 
loss, the user system 16 will immediately generate a lost frame message to the server 
10 10. 

The server 10 interprets the lost frame message as an instruction to encode the 
next frame as an independent frame 56 and will provide for the codec 38 to encode the 
next frame as an independent 56. Again, while any freeze frame caused by loss of 
frames during the recording session will still remain when viewed on the user system 
15 16, any additional freeze frame caused by frame loss during the play back session will 
be limited to the time duration required between when the user system 16 to detects 
frame loss and when the independent frame 56 generated by the server is transferred to 
the user system 16. 

The fourth frame loss embodiment includes storing the recording sequence of 
20 compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. The video mail file 22 will be missing any 
frames that were lost during the recording session. During playback, the recording 
sequence of compressed images 52 is retrieved from the video mail file 22 and 
simultaneously transferred to both the delay circuit 24 and to the codec 38 for decoding. 
25 The delay circuit 24 delays the recording sequence of compressed images 52 for 

a time duration 126 such that the an image frame output from the decoder 44 
corresponds to the compressed frame output from the delay circuit 24. The output of 
the delay circuit 24 (e.g. the recording sequence of compressed images 52) is 
transferred to the user system 16 during the playback session. However, if the user 
30 system 16 detects any frame loss, the user system 16 will immediately generate a lost 
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frame message to the server 10. The server 10 interprets the lost frame message as an 
instruction to encode the next frame as an independent frame 56 and provides for the 
encoder 46 to encode the next frame output by the decoder 44 as an independent 
frame 65 and provides for the media interface 42 to substitute such independent frame 
5 65 for its corresponding frame from the delay buffer 24. Again, while any freeze frame 
caused by loss of frames during the recording session will still remain when viewed on 
the user system 16, any additional freeze frame caused by frame loss during the play 
back session will be limited to the time duration required between when the user system 
16 to detects frame loss and when the independent frame 56 generated by the server is 

10 transferred to the user system 16. 

The fifth frame loss embodiment includes converting the recording sequence of 
compressed images 52 to the robust format prior to storage. There exist two systems 
for implementing this fifth frame loss embodiment. In the first system, the server 10 
may periodically generate a lost frame message to the caller system 14. In response, 

15 the caller system will generate its next compressed image frame as an independent 
frame. The server 10 will provide this periodic lost frame message to the caller system 
14 regardless of whether any frame really has been lost. The purpose of the request is 
to assure that the recording sequence of compressed images 52 is in the robust format 
when received by the server 10. 

20 In the second system, the server 1 0 may generate the robust format locally. 

More specifically, the server 10 may transfer the recording sequence of compressed 
images to the codec 38 for decoding and re-encoding in the robust format and 
transferred to storage 28 as the video mail file 22. During playback, each compressed 
frame from the video mail file 22 is transferred to the user system 16. 

25 A first sub embodiment of this fifth embodiment is the robust format wherein at 

least one independent frame 56 is encoded for each fixed time period. A second sub 
embodiment of this fifth embodiment is the robust format wherein a fixed time period 
exists between each independent frame. Again, while any freeze frame caused by loss 
of frames during the recording session will still remain when viewed on the user system 

30 1 6, any additional freeze frame caused by frame loss during the play back session will 
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be limited to only the time period between the lost frame and the when next 
independent frame 56 is received. 

A sixth frame loss embodiment includes establishing the first internet protocol 
channel 18 as a TCP/IP channel such that the IP services module 32 will operate 
5 TCP/IP error correction protocols to ensure that there are no lost frames during the 
recording session. The recording sequence of compressed images 52 is stored as the 
video mail file 22 in the storage 28. During playback, the recording sequence of 
compressed images 52 is retrieved from the video mail file 22 and provided to the user 
system 16 using any of the playback systems of the first through the fourth 

10 embodiments discussed above. 

A seventh frame loss embodiment includes establishing the second internet 
protocol channel 20 as a TCP/IP channel such that the IP services module 32 will 
operate TCP/IP error correction protocols to ensure that there are no lost frames during 
the playback session. However, the first internet protocol channel 18 over which the 

15 recording sequence of compressed images is transferred remains a UDP/IP channel. 
The recording sequence of compressed images 52 is stored as the video mail file 22. 
During playback, the recording sequence of compressed images 52 is retrieved from 
the video mail file 22 and provided to the user system 16 over the TCP/IP second 
internet protocol channel 20. Again, while freeze frame caused by loss of frames during 

20 the recording session will still remain when viewed on the user system 16, no additional 
freeze frame will be caused during the play back session because of the error correction 
protocols of TCP/IP. 

An eighth embodiment includes establishing both the first internet protocol 
channel 18 and the second internet protocol channel 20 as a TCP/IP channels. As such 

25 the IP services module 32 will operate TCP/IP error correction on both channels 18 and 
20 to ensure that there are no lost frames during either the recording session or the 
playback session. 

Signaling Client. 

30 The ladder diagram of Figure 2a represents interaction between the server 10 
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and the caller system 14. Referring to Figure 2a in conjunction with Figure 1 , steps 68 
represent the signaling module 34 establishing the first internet protocol channel 18 with 
the caller system 14. As discussed above, the SIP signaling described herein is for 
purposes of illustration only and is not intended to limit the scope of the invention to only 
5 those systems that implement SIP. 

Step 70 represents the signaling module 34 receiving an invite message from the 
caller system 14 (or a proxy representing the caller system 14). Step 72 represents the 
signaling module 34 returning a ringing message. The purpose of the ringing message 
72 is to trigger the caller system 14 to generate ringing tones for the caller during any 

10 time period needed before the signaling module is able to provide an OK message at 
step 74. The OK message is a signal to the caller system 14 that the server 10 can 
accept the call and is ready to begin the exchange of media capabilities necessary to 
receive the recording sequence of compressed frames 52 over the first IP channel 18. 
Following the OK message, at step 78, the caller system 14 provides an 

15 acknowledge message at step 76. The signaling client 32 then exchanges media 
capabilities and port selection for the first IP channel 18 and for a real time control 
channel 80. In any of the first through fifth, and the seventh, embodiments discussed 
above, the first IP channel 18 will be a UDP/IP channel over which the recording 
sequence of compressed images will be transferred. In the sixth and eighth 

20 embodiments, the first IP channel 18 will be a TCP/IP channel. 

The ladder diagram of Figure 2b represents interaction between the server 10 
and the user system 16. Referring to Figure 2b in conjunction with Figure 1 , steps 82 
represent the signaling module 34 establishing the second internet protocol channel 20 
with the user system 20. 

25 Step 84 represents the signaling module 34 receiving an invite message from 

the user system 16 (or a proxy representing the user system 16). Step 86 represents 
the signaling module 34 returning a ringing message. Step 88 represents sending an 
OK message to the user system 16 indicating that the server 10 can accept the call and 
is ready to begin the exchange of media capabilities necessary to transfer the playback 

30 recording session sequence of compressed frames 58 over the second IP channel 20. 
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Following the OK message, at step 88, the user system 16 provides an 
acknowledge message at step 90. The signaling client 34 then exchanges media 
capabilities and port selection for the second IP channel 20 and for a real time control 
channel 94. In any of the first through sixth embodiments discussed above, the second 
5 IP channel 20 will be a UDP/IP channel over which the recording sequence of 

compressed images will be transferred. In the seventh and eighth embodiments, the 
second IP channel 20 will be a TCP/IP channel. 

Media Interface 

10 The first embodiment of operation of the media interface is illustrated with 

referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 
diagram of Figure 5a in conjunction with Figure 1 . The flow chart of Figure 3a 
represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4b represents operation of the playback module 50 of the media 

15 interface 42, and the frame diagram 5a represents the play back sequence of image 
frames 58. 

With reference to Figure 3a and Figure 1 , the recording module receives each 
frame 54 and 56 of the recording sequence of compressed image frames 58 from the 
caller system 14 over the first IP channel 18, as is represented by step 126, and stores 
20 each received frame 54 and 56 as the video mail file 22 in the storage. The endless 
loop between steps 126 and 128 represents receipt and storage of each frame 54 and 
46 in the sequence. 

With reference to Figure 4b and Figure 1 , the playback module simultaneously 
operates three processes. The first process includes obtaining the recording sequence 
25 of compressed image frames 52 (as stored by the recording module 48) from the video 
mail file 22 and transferring each compressed image (in sequence) to the decoder 
module 44 as is represented by steps 100 and 102. 

The decoder 44 receives the compressed image sequence and decodes to 
generate motion video images 40 which are immediately passed to the encoder 46. 
30 The encoder 46 generates the play back sequence of compressed image frames 58 as 
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is shown in Figure 5a. 

The second process includes obtaining the playback sequence of compressed 
image frames 58 from the encoding module 46 and transferring the playback sequence 
of compressed images frames 58 to the user system 16 over the second IP channel 20 
5 as is represented by steps 104 and 106. 

The third process includes determining whether an event has occurred at steps 
108 and, if so, generating a signal to the encoder module 46 to generate the next frame 
in the playback sequence of compressed image frames 58 as an independent frame 54. 

In the first embodiment, the event that would trigger such a signal would be an 
10 elapse of time during which an independent format frame 54 has not been received 
from the decoder module 46. 

Referring to Figure 5a, the resultant playback sequence of compressed image 
frames 58 includes at least one independent frame for each fixed time duration 62. 

The second embodiment of operation of the media interface is illustrated with 
15 referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 

diagram of Figure 5b in conjunction with Figure 1 . Similar to the first embodiment, the 
flow chart of Figure 3a represents operation of the recording module 48 and the 
flowchart of Figure 4b represents operation of the playback module 50. Figure 5b 
represents the play back sequence of image frames 58. 
20 The difference between the first embodiment and the second embodiment is that 

in the process of determining whether an event that would trigger a signal to generate 
an independent frame is not an elapse of time during which an independent frame has 
not been received, but an elapse of a time period 64 since the last signal to generate an 
independent frame was provided to the encoding module 46. 
25 Referring to Figure 5b, the resultant playback sequence of compressed image 

frames 58 includes an independent frame 54 with a fixed time period 64 there between. 

The third embodiment of operation of the media interface is illustrated with 
referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 
diagram of Figure 5c in conjunction with Figure 1 . Similar to the first and second 
30 embodiments, the flow chart of Figure 3a represents operation of the recording module 
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48 and the flowchart of Figure 4b represents operation of the playback module 50. 
Figure 5c represents the sequence of frames during playback. 

With reference to Figure 4b and 5c, the playback module simultaneously 
operates three processes. As discussed above, the first process (steps 100 and 102) 

5 includes obtaining the recording sequence of compressed image frames 52 (as stored 
by the recording module 48) from the video mail file 22 and transferring each 
compressed image (in sequence) to the decoder module 44. 

The decoder 44 receives the compressed image sequence and decodes to 
generate motion video frames 40 which are immediately passed the encoder 46. 

10 Because of the time delay for decoding, Figure 5c shows the motion video frames 40 
delayed from the recording sequence of compressed image frames 52. 

The encoder module generates the play back sequence of compressed image 
frames 58. Because of the time delay for encoding, Figure 5c shows the play back 
sequence of compressed image frames 58 delayed from the motion video frames 40 

15 and further delayed from the recording sequence of compressed image frames 52. 
The second process (steps 104 and 106) includes obtaining the playback 
sequence of compressed image frames 58 from the encoding module 46 and 
transferring the playback sequence of compressed images frames 58 to the user 
system 16 over the second IP channel 20 as is represented by steps 104 and 106. 

20 The third process includes determining whether an event has occurred at step 

108 and, if so, generating a signal to the encoder module 46 to generate the next frame 
in the playback sequence of compressed image frames 58 as an independent format 
frame 54. In this third embodiment, the event that would trigger such a signal would be 
receipt of a lost frame message 36 from the user system 16 indicating that a frame loss 

25 has occurred. As such, the resultant playback sequence of compressed image frames 
58 includes an independent frame 54 following each lost frame message 36. 

The fourth embodiment of operation of the media interface is illustrated with 
reference to the flow chart of Figure 3a, the flowchart of Figure 4c, and the frame 
diagram of Figure 5d in conjunction with Figure 1 . Similar to the previously discussed 

30 embodiments, the flow chart of Figure 3a represents operation of the recording module 
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48 and the flowchart of Figure 4c represents operation of the playback module 50. 
Figure 5d represents the sequence of frames during playback. 

With reference to Figure 4c and 5c, the playback module simultaneously 
operates two processes. The first process (steps 112 and 1 14) includes obtaining the 
5 recording sequence of compressed image frames 52 (as stored by the recording 
module 48) from the video mail file 22 and transferring each compressed image (in 
sequence) to both the decoder module 44 and to the delay buffer 24. 

The decoder module 44 receives the compressed image sequence and decodes 
each frame to generate motion video frames 40. Because of the time delay for 
10 decoding, Figure 5d shows the motion video frames 40 delayed from the recording 
sequence of compressed image frames 52 by a time period 128. 

The delay buffer 24 delays the playback sequence of compressed image frames 
52 by a time period 126 that corresponds to the period of time 128 required to generate 
the motion video frames 40 plus a period of time 130 required to re-encode a motion 
15 video 40 to generate an independent frame 54. 

In the second process, the playback module 50 sequentially retrieves each 
compressed image from the delay buffer 24 at step 1 16. Step 118 represents 
determining whether a lost frame message 36 has been received from the user system 
16. If a lost frame message 36 has not been received, the compressed image frame is 
20 transferred to the user system 16 over the second IP channel 20 at step 124. 

Alternatively, if a lost frame message 36 has been received, an instruction is 
passed to the encoder 46 to generate an independent frame 54. Step 120 represents 
obtaining the independent frame 54 from the encoder 46 as a lost frame correction 
frame 68. Step 1 22 represents substituting the lost frame correction frame 68 for a 
25 frame in the sequence of frames received from the delay buffer 24. The substituted 
frame is then provided to the user system 16 at step 124. Again, the endless loop 
represents such processing of each frame in the sequence. 

The resultant playback sequence of compressed image frames 58 includes an 
independent frame 54 following each lost frame message 36. 
30 The fifth embodiment of operation of the media interface is illustrated with 
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referring to the flow chart of Figure 3b, the flowchart of Figure 4a, and the frame 
diagram of Figure 5a in conjunction with Figure 1 . The flow chart of Figure 3b 
represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4a represents operation of the playback module 50 of the media 
5 interface 42, and the frame diagram 5a represents the play back sequence of image 
frames 58. 

With reference to Figure 3b, the recording module simultaneously operates three 
processes. The first process includes obtaining a each frame of the recording 
sequence of compressed image frames 52 over the first IP channel 18 and passing the 
10 frame to the decoder module 44 at steps 130 and 132 respectively. Again, the endless 
loop represents repeating this process for each frame in the sequence of compressed 
image frames 52. 

The decoder 44 receives and decodes each compressed image frame and 
generates motion video frames 40 which are immediately passed to the encoder 46. 
15 The encoder 46 encodes each video image frame 40 to generate the play back 

sequence of compressed image frames 58 as is shown in Figure 5a. 

The second process includes obtaining each compressed frame of the playback 
sequence of compressed image frames 58 from the encoder 46 and transferring the 
compressed frame to the video mail file 22 at steps 134 and 136 respectively. Again, 
20 the endless loop represents repeating this process for each frame of the playback 
sequence of compressed images frames 58. 

The third process includes determining whether an event has occurred at step 
138 and, if so, generating a signal to the encoder module 46 to generate the next frame 
in the playback sequence of compressed image frame 58 as an independent format 
25 frame 54. In this fifth embodiment, the event that would trigger such a signal would be 
an elapse of time during which an independent format frame 54 has not been received 
from the decoder module 46. 

With reference to Figure 4a, the play back module simply obtains the sequence 
of compressed images from the video mail file 22 at step 96 and transfers such 
30 sequence to the user system 16 as the playback sequence of compressed images 58 at 
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step 98. 

Referring to Figure 5a, the resultant playback sequence of compressed image 
frames 58 includes at least one independent frame for each fixed time duration 62. 
The sixth embodiment of operation of the media interface is illustrated with 
5 reference to the flow chart of Figure 3b, the flowchart of Figure 4a, and the frame 
diagram of Figure 5b in conjunction with Figure 1 . Similar to the fifth embodiment, the 
flow chart of Figure 3b represents operation of the recording module 48 and the 
flowchart of Figure 4a represents operation of the playback module 50. Figure 5b 
represents the play back sequence of image frames 58. 

10 The difference between the sixth embodiment and the fifth embodiment is that in 

the process of determining whether an event that would trigger a signal to generate an 
independent frame is not an elapse of time during which an independent frame has not 
been received, but an elapse of a time period 64 since the last signal to generate an 
independent frame was provided to the encoding module 46. 

15 The seventh embodiment of operation of the media interface is illustrated with 

referring to the flow chart of Figure 3c, the flowchart of Figure 4a, and the frame 
diagram of Figure 5b in conjunction with Figure 1 . The flow chart of Figure 3c 
represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4a represents operation of the playback module 50 of the media 

20 interface 42, and the frame diagram 5b represents the play back sequence of image 
frames 58. 

With reference to Figure 3c, the recording module receives each frame 54 and 
56 of the recording sequence of compressed image frames 58 from the caller system 14 
over the first IP channel 18, as is represented by step 1142, and stores each received 
25 frame 54 and 56 as the video mail file 22 in the storage at step 144. At step 146, the 
recording module determines whether a time period - on the order of one second - has 
elapsed. If the time period has not elapsed, the endless loop between back to step 142 
represents receipt and storage of each frame 54 and 46 in the sequence. 

After the time period has elapsed at step 146, the recording module generates a 
30 lost frame message to the caller system 14. In response to lost frame message, the 
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caller system 14 will encode the next frame in an independent format. 

As such, the recording sequence of compressed images 52, as received by the 
recording module will be in the robust format of Figure 5b and the playback module may 
utilize the process describe with respect to Figure 4a for presenting the play back 
5 sequence of compressed images 58 to the user system 1 6. 

The eighth embodiment of operation of the media interface (which corresponds 
to the sixth frame loss embodiment of the present invention) is illustrated with reference 
to the flow chart of Figure 3a and the flow chart of Figure 4a. Because both the first 
internet channel 18 and the second internet channel 20 are established over TCP/IP 
10 connections, no schemes for increasing the number of independent frames 54 within 
the play sequence of compressed images 58 are needed. 

In an ninth embodiment of operation of the media interface (which corresponds to 
the seventh frame loss embodiment of the present invention), includes operation of 
each of the first through fourth embodiments of operation of the media interface 42. 

15 

Summary 

It should be appreciated that the systems and methods discussed herein provide 
for video mail server to provide video mail to a user device with reduced frame loss and 
therefore reduced freeze time. 

20 Although the invention has been shown and described with respect to certain 

exemplary embodiments, it is obvious that equivalents and modifications will occur to 
others skilled in the art upon the reading and understanding of the specification. For 
example, the exemplary embodiments discussed herein operate utilizing compression 
techniques of the H.263 protocol. It is readily apparent to those skilled in the art that the 

25 teachings of the present invention may also be implemented on other compression 
protocols that utilize interdependent frame compression techniques. The present 
invention includes all such equivalents and modifications, and is limited only by the 
scope of the following claims. 

30 
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